The image pickup element which is mounted on a digital imaging apparatus such as a digital camera and a cellular phone with a built-in camera includes such as a charge coupled type image pickup element (a CCD type image pickup element) and a metal oxide semiconductor type image pickup element (a MOS type image pickup element). Further, the MOS type image pickup element includes a complementary metal oxide semiconductor type image pickup element (a CMOS type image pickup element), an N-channel metal oxide semiconductor type image pickup element (an NMOS type image pickup element), and the like. In recent years, there is a tendency that the number of pixels of those image pickup elements has been increased, and each of them has been progressed as a high definition image pickup element. The CCD type image pickup element has characteristics that its dynamic range is wide with less noise, and the MOS type image pickup element has characteristics that a simple structure is achieved because of using an MOS process with a single power source, which is suitable for high resolution.
Next, a signal processing for imaging by using the digital imaging apparatus will be schematically explained.
An example of signal processing related to a still image pickup of one frame, namely, a single image pickup, in the digital imaging apparatus is shown. Light from an object forms an image at a light receiving unit of the image pickup element, and a number of pixels arranged on the light receiving unit accumulate charges of a certain quantity according to a light intensity. For one line of the light receiving unit, accumulated charges of pixels are read out pixel-by-pixel through an output unit of the image pickup element as analog signals, which are then converted into digital pixel signals (RAW data) in an analog/digital conversion unit, and is temporarily stored in a buffer memory such as a synchronous DRAM (SDRAM). When reading of the one line, A/D conversion, and writing into the SDRAM, etc, are completed, processing from the reading to the writing is similarly repeated for a second line, a third line, . . . , and a final line, and data of one frame of the image (one image pickup) is temporarily stored in the SDRAM. Next, a signal processing arithmetic operation such as a zoom processing of magnification/reduction is performed for the temporarily stored data, namely, RAW data, and the data after operation is temporarily stored in the SDRAM again. Next, by appropriately processing the data by using a processor, the data is converted into the data (compressed data) of a compressed data format such as JPEG being suitable for storing the data, and thereafter is temporarily stored in the SDRAM again. Then, the data is read out from the SDRAM at high speed by a direct memory access (DMA) control, etc., and the read data is stored in an external semi-permanent storage memory. Here, the semi-permanent storage memory may be a storage medium generally used as an image recording medium of a digital camera such as an SD Memory Card.
By continuously executing the signal processing related to the aforementioned single image pickup, a continuous image pickup, namely, a consecutive image pickup is realized. However, the processing from the RAW data to the data (compressed data) of the compressed data format is a time-consuming processing compared to the processing of reading out the accumulated charges of the pixel and temporarily storing it as the RAW data. Therefore, in case of a continuous shooting, the processing of storing the RAW data in the SDRAM, and the processing of converting the RAW data into the compression data are performed simultaneously, and the read out RAW data is temporarily stored in the SDRAM additionally as far as a storage capacity of the SDRAM allows. Accordingly, in order to increase the number of frames capable of continuous shooting, the storage capacity of the SDRAM needs to be increased.
In addition, along with an increase of the number of pixels of the image pickup element of recent years, a data size of the RAW data for one frame of an image is also increased. Therefore, when the storage capacity of the SDRAM is limited to a degree being same as that of a conventional product, the number of frames capable of continuous shooting is decreased along with the increase of the number of pixels of the image pickup element. Therefore, when a high definition of the imaging apparatus is realized by increasing the number of pixels of the image pickup element, the storage capacity of the SDRAM simultaneously needs to be made larger and the number of frames capable of continuous shooting needs to be secured. In addition, when the data size of the RAW data becomes large, the SDRAM capable of being accessed at higher speed than conventional is desired. However, a larger storage capacity of the SDRAM and a higher speed of access are disadvantageous in terms of its cost.
Conventionally, techniques for solving the aforementioned problem are proposed.
FIG. 11 is a block diagram illustrating a method for reducing a memory usage of a frame memory being disclosed in Patent Document 1, with a digital still camera taken as an example.
First, a configuration of the digital still camera of the Patent Document 1 will be explained. A digital still camera 100 includes: an image processing unit (CPU) 110; a flush memory 120; a JPEG-LSI 130; a display/capture controller 141; a buffer memory 142; a memory transfer controller 143; an address bus switching unit 144; a read data latch 145; an output level latch 146; a difference decompression/adder 147; a difference compression/decompression conversion table 148; a subtraction/difference compression unit 149; a write data latch 150; an input level latch 151; a frame memory 160; an image output unit 170; a image input unit 180; and a compression/decompression unit 140.
Next, an operation of the digital still camera 100 of the Patent Document 1 will be explained. The data before compression inputted from the image input unit 180 of FIG. 11 is sent to the subtraction/difference compression unit 149. Subtraction and difference compression are performed therein. At that time, the compression is performed with reference to the difference compression/decompression conversion table 148. The data after compression being obtained as a result is temporarily stored in the frame memory 160 through a data bus. Data before decompression being compressed that exists in the frame memory 160 is converted into the data after decompression with reference to the difference compression/decompression conversion table 148 in the difference decompression/adder 147, and is outputted from the image output unit 170.
This method needs to have the difference compression/decompression conversion table 148 in the ROM, or the like, for performing the compression of a difference value of the data. A circuit scale becomes smaller compared with a conventional method. Even so, it is inevitable to use the ROM, thus making a circuit structure larger, and a processing load is still large.
FIGS. 12 and 13 are schematic diagrams illustrating a method of irreversible compression encoding of the digital signal disclosed in Patent Document 2. FIG. 12 is a block diagram of a single board CCD, and FIG. 13 is a flowchart of the irreversible compression encoding. FIG. 12 shows a target pixel x, same color pixels f, e, d which are to be processed prior to the target pixel x, and different color adjacent pixels c, a, b being adjacent to the pixel x.
Next, the processing of a compression encoding method disclosed in the Patent Document 2 will be explained with reference to FIG. 13. This method is a method of performing an entropy encoding to a predicted error between a predicted value y by an optimum prediction expression and a value of a target pixel x in a single board CCD on which color filters R, G, B are arranged, so that the image data may be compressed. The method includes: calculating a predicted value by using the pixel value of a nearby pixel and the pixel value of the adjacent pixel on which a color filter of a color component different from that of the target pixel is arranged; calculating a predicted value by using the pixel value of the nearby pixel and a same color pixel on which the color filter of the same color as the target pixel is arranged; determining which of the predicted values is closer to the target pixel x; and deciding which a pixel value of the adjacent pixel or the same color pixel is to be used to calculate a predicted value of the next target pixel based on this determination result.
In this method, quantization is performed with a nonlinear table, a constant value is uniformly multiplied, and a table value to be used in an actual operation is calculated. Thus, compressibility can be changed in a irreversible conversion. The quantized data is further entropy-encoded.
In the method disclosed in the Patent Document 2, the memory such as a ROM is also required in quantizing the predicted error Δ being calculated from a pixel value by using a predetermined quantization table, thus enlarging a circuit structure and increasing a processing load.
FIG. 14 is a block diagram of an image encoding device disclosed in Patent Document 3. FIG. 14 shows the image encoding device in which an input pixel value of a dynamic range of d-bits is inputted from a pixel value input unit 101 and the input pixel value is encoded and converted into a quantized value of n-bits and the quantized value is then outputted from an output unit 105.
This image encoding device further includes: predicted value generating means 106 for generating a predicted value for the input pixel value; linear quantizer generating means 102 for generating, in d-bit accuracy, a linear quantizer having a quantization width set at (d−n)-th power of 2 and linear quantization representative points of the number which is obtained by subtracting an additional upper limit number being preliminarily set from n-th power of 2; and nonlinear quantizer generating means 103 for generating the nonlinear quantizer having a quantization width in the vicinity of the predicted value being set smaller than that of the linear quantizer by adding linear quantization representative points of the number which is less than the upper limit number to the linear quantizer in the vicinity of the predicted value. In the image encoding device, a quantization unit 104 quantizes the input pixel value by using the nonlinear quantizer which is generated in the nonlinear quantizer generating means 103 and a quantized value being obtained is outputted.
The image encoding device disclosed in Patent Document 3 has no ROM table for quantization. In this respect, this image encoding device is expected to be realized as a smaller-scaled apparatus than the apparatus of the invention disclosed in Patent Documents 1 and 2.
Patent Document 1: JP 11-341288 A
Patent Document 2: JP 2000-244935 A
Patent Document 3: JP 10-056638 A